911 services and vital sign measurement utilizing mobile phone sensors and applications

ABSTRACT

Improved methods for utilizing 911 services, for implementing 911 dispatch protocols, and for measuring vital signs of a human, all by accessing mobile phone sensors and applications, are disclosed. Vital signs such as heart rate, breathing rate, breathing distress, and blood pressure can be measured using mobile phone sensors and applications. A method for differential estimation of blood pressure involves the synchronization of time between two mobile phones, locating an appropriate position for one cell phone and recording heart sounds, and recording video data from the finger tip of the subject using the other mobile phone.

This application claims priority to U.S. Provisional Patent ApplicationSer. No. 61/537,382, entitled 911 SERVICES AND VITAL SIGN MEASUREMENTUTILIZING MOBILE PHONE SENSORS AND APPLICATIONS, filed on Sep. 21, 2011,the entire content of which is hereby incorporated by reference.

BACKGROUND

This disclosure pertains to improved methods for utilizing 911 servicesand for implementing 911 dispatch protocols by accessing mobile phonesensors and applications. This disclosure also pertains to measurementof vital signs of a patient using mobile phone sensors and applications,including estimation of blood pressure without a traditional stethoscopeand cuff.

The United States 9-1-1 System has its origin in 1958, when theCommission on Law Enforcement and Administration of Justice suggestedreplacing the patchwork of local police and fire numbers with a singlenational emergency number. The dispatcher is an important player in the911 call center. This is the person who takes information about theemergency and arranges adequate resources to be sent to handle thesituation. The process of gathering information by the dispatchers toaccess the emergency situation has been standardized into documentscalled Dispatch Protocols. The first use of the standardized protocolswas recorded in Arizona in 1975.

Table 1 below shows the data for various categories of calls received inpercentage terms. The table shows that Medical emergencies account forthe largest number of calls, followed by fire and vehicle accidents. TheOther category has the largest number of calls, but includes calls thatcannot be classified into any category. This number also includes callsthat were considered non-emergency

TABLE 1 Category of 911 Calls Type of Emergency Call Percentage of TotalCalls Medical Emergency 37 Fire 11 Vehicle/Accidents 10 Chemical Hazards2 Floods/Water Damage 1 Electricity/Wire Down 1 Other 38

Emergency dispatch protocols were based in legacy telecom networks whereonly voice calls were used and the networks were all based on dedicatedlandlines over a call connection. Over the years the technology hasdeveloped to include wireless networks in which the call connections areno longer dedicated landlines, but based on packet networks. Thebandwidth of the networks has also increased to a point where multimediaservices are available to the general public.

Emergency response services are provided in most of the advancedcountries. The role of the emergency dispatcher has its origins in theUnited States, but it has gained acceptance in all parts of the world.In the US, the Dispatch Protocols are called the Medical PriorityDispatch System (“MDPS”). It is a system that has about 37 cards andeach card gives instructions to the dispatcher for a specific emergencytype. A similar Dispatch Protocol is used by the state of New Jersey andis called the Emergency Medical Dispatch Guidecards. This protocol alsohas a set of cards that the dispatcher uses based on the type ofemergency call. These cards act as a guide to the dispatcher. Thesuccess of the notion of Dispatch Protocols can be gauged from the factthat today many developed countries have developed their own protocols.The UK uses AMPDS, while France uses its own version of DispatchProtocols, called SAM U.

The 9-1-1 system in the U.S. has evoked to include, for example,wireless 9-1-1 services in 1998. In 205, the FCC mandated that Voiceover IP (“VoIP”) providers must offer 9-1-1 services also. Today, thearchitecture for Next Generation 9-1-1 (“NG911”) has been designed bythe National Emergency Number Association (“NENA”). NENA firstidentified the need for an overhaul of the 911 system in the year 2000.The first document describing the future path was produced in 2001 andby the end of 2003 the standards development activity had started. NENAis developing several documents relating to the architecture. Some ofthe documents are already complete. For example the overall requirementsdocument is defined in NENA i3 Technical Requirements Document,available publicly.

Each year, about 200 million emergency calls are placed in the US, withabout one third originating from mobile phones. With the deployment ofNG911, there is the potential to better utilize the capabilities ofthese mobile phones to improve 911 services and dispatch protocols.

Measurement of vital signs in a human is an arduous task when suddendizziness or fainting occur during unexpected situations. Theseoccurrences are the most common symptoms of low blood pressure. Bloodpressure, the amount of force applied on the walls of the arteries whenthe blood is forced throughout the body, depends on factors such as theamount of blood in the body, the pumping rate of the heart, theflexibility of the arterial walls, and the resistance to blood flow dueto the size of the arteries. The blood pressure of a human variescontinuously due to physical activity, medication, anxiety, andemotions. The body has unique mechanisms to regulate a person's bloodflow; whenever a person's blood pressure drops, the heart rate increasesto pump more blood and the arterial walls contract to increase the bloodpressure. Blood pressure is given by two numbers measured in millimetersof mercury (mmHg). The first number is the systolic pressure, whichrepresents the amount of pressure applied in the arteries when the heartcontracts. The second number is the diastolic pressure, which representspressure on the arteries when the heart is at rest.

Blood pressure is typically measured in the upper arm with the personcomfortably seated and the arm in level with the heart. The measurementapparatus is a mercury sphygmomanometer comprising a manometer, pressurecuff, bladder and a gauge to show the pressure value inside the cuffwhich is wrapped tightly around the upper arm about an inch from theelbow since the upper arm is closest to the heart and therefore errorsdue to upflow/downflow of blood are eliminated. When this setup is done,the cuff is inflated to a pressure of about 210 mmHg or usually 20 to 30mmHg above the normal systolic pressure. The gradual reduction ofinflation causes a turbulent flow of blood, creating a sound. The cuffpressure corresponding to the first sound produced is taken as thesystolic pressure. The cuff pressure is further reduced and when thesounds due to blood flow cannot be heard with the stethoscope. That cuffpressure is taken as the diastolic pressure.

However, to check a person's blood pressure during unexpectedsituations, there is a need for a portable, convenient device orapparatus. Despite the availability of digital wrist and armblood-pressure meters, most people do not carry these devices duringtheir daily commute to work place, gymnasiums, recreating facilities, orother places.

However, smartphones today have become increasingly popular with thegeneral public for their diverse functionalities such as navigation,social networking, and multimedia facilities. These phones are equippedwith high end processors, high resolution cameras, and built-in sensorssuch as accelerometer, orientation sensor and light-sensor. It isestimated that 25.3% of US adults use smart phones in their daily lives.Being prevalent in people's lives, it is highly desirable to utilizethese mobile phones to measure vital signs of a human.

SUMMARY

The present disclosure relates generally to improved 911 services,emergency dispatch protocols, and vital sign measurement, includingcuff-less differential estimation of blood pressure, utilizing mobilephone sensors and applications.

Today, most emergency calls are based on the Public Switched TelephoneNetwork (“PSTN”). However, there were approximately 80 million Voiceover IP (“VoIP”) subscribers worldwide in 2007, and 50 percent of globaltelecommunications traffic is currently handled over IP networks.Although a small percentage of emergency calls are currently placed byVoIP subscribers, existing emergency calling systems must supportIP-based emergency calls. To support IP-based emergency communicationsand the variety of new services that it allows, a new architecture iscurrently being developed and tested: Next Generation 9-1-1 (NG-9-1-1).

Telecom networks are moving toward Voice over IP protocols. Over time,an increasing number of calls will be made over these networks asopposed to the legacy networks. While these advances allow access tomultimedia communications in everyday life, it also raises severalissues that need to be resolved. Specifically for 911 calls, there arenetwork related issues, such as identification of caller location (Song,W. et al. 2008). The challenges for these issues include:

Remote Media Control: Automatic remote control of cameras in the mobilesto change focus, lighting, contrast, Codecs, bandwidth, and such, tohelp prepare the call taker to better respond to the emergency.

Emergency Dispatch Protocols: Modification of Emergency DispatchProtocols to take advantage of this new technology so as to reduce thenumber of questions asked over the phone and similarly reduce the numberof instructions given.

High: Quality of Service and traffic management need to be designed sothat the 9-1-1 calls are not disrupted. This has always been true, butwith multi-media services this issue also needs to be addressed.

Human Machine Interface (HMI): HMI is another very important researchissue. The video screens at the call center will need a new design.There may be need for multiple screens or multiple windows on onescreen. This includes requests, responses, and usability of screens onthe operator side and also design of controls on the caller side.

Connection Management: Connection management becomes an important issueto be addressed as several responders may be sharing video and audiostreams.

Security: Security of the NG9-1-1-network not only needs to bemaintained but also enhanced.

Privacy: Study image distortion and masking of parts of video or imagesto protect privacy of callers.

Social Networks as First Responders: Many times people call friends andfamily members first when faced with a problem. The use of socialnetworks has made this an important addition to emergency responsesystem. This process can make friends and family first responders in anemergency situation.

Medical Records: Provide the ability to select which responder orresponders can have access to medical or personal information of thecaller(s).

The next generation of communication system will have the capability ofmultimedia transmission and several streams being broadcastedsimultaneously. FIG. 1 shows an example of such a multimedia based callscenario. As shown in FIG. 1 the smart phone running VOW software calledSipdroid has several sensors and applications allowing video, picturesand voice transmission.

The call center may have four categories of calls for a given incident.First are individual callers. These scenarios are based on individualscalling 9-1-1 help. These individuals could range from children toelderly, English and non-English speaking, handicapped, disoriented orbarely alive. Individual callers can use the traditional landline phone,cell phone or the IP phone to make a 911 call. In the case of medicalemergency, new technologies can provide additional useful information tothe Public Safety answering Point (“PSAP”) operator to better handle theemergency. For example a caller can use the video camera on his cellphone to show the nature of emergency. Instead of a verbal description,the PSAP operator can visually see the problem. A caller who ishandicapped and cannot speak can use text messaging or also the cellphone camera (image or video) to show the problem. This can betterassist in a two way communication as the PSAP can visually see if thecaller is following the instructions given by the operator.

Second are Third Party Services. These scenarios are based on calls madeby Services like OnStar that monitors vehicle crashes, alarm companiesthat monitor break-ins, fires or other emergencies, home monitoringservices of medical devices for people, and services that monitorinfrastructures like highways, bridges and water ways. These servicesare dependent on sensors alerting them about the problem. Many times thealert by the sensors is followed by a call to some person to confirm theemergency, for example a call to the home owner in case of a break-inalert. These services in turn then call the appropriate 911 service tohandle the emergency. While the advantage of these services is that itcuts down on the number of false alarms, the disadvantage is that italso causes a delay in handling the emergency. In this case also newtechnology can help. Direct feed of a video camera to the PSAP wouldhelp. The third party operators should be able to route a video feed to911 PSAP.

Third are calls from Emergency Response Units in the field. Thesescenarios are based on situations where the units that respond toemergency require additional support. Some of the examples of thesesituations are when medical personnel responding to an emergencysituation need support of law enforcement personnel if the situationturns dangerous, or fire station responders might discover some illegalactivities or police officers responding to a vehicle crash might callmedical services for help. When follow up units are responding to acrisis, video technology can be used to relay images of the scene to allunits. The PSAP operator can better coordinate the response ofindividual units if there is a video display at PSAP as well as adisplay with each unit.

Finally, the fourth category is automatic calls based on sensors. Theseare scenarios where a fusion of data from different sensors results inan automatic 9-1-1 call. One possible scenario could be sensors detectchemical spills and automatically notify emergency services. Anotherexample could be vehicle sensors transmitting information about thestate of the vehicle (rollover or not).

The network architecture will have to consider not only the new VoIPscenarios, but also needs to be backward compatible. The legacy networksand the wireless networks will also be operational in severaljurisdictions simultaneously. So a 911 call may go over one or more ofthese networks before reaching its destination PSAP.

The increase in bandwidth of telecommunications networks allows thepossibility of several streams of data simultaneously transmittinginformation in real time. On the other hand the sensor technology hasmatured to a point where they are embedded in cell phones and otherdevices. The mass production of smart phones with built-in sensors suchas an accelerometer, a camera with flash and a microphone sensor makesthem valuable to the biomedical domain. It is possible for these sensorsto automatically transmit information that they detect about a specificmedical condition. These sensors may be used in dedicated medicaldevices that people can wear on their person to help them diagnose aspecific medical condition. For example there are devices that helppeople track their daily blood pressure. Or the sensors may be embeddedin more general personal devices like the cell phones. In emergencies,the callers may be physically or cognitively impaired and thesededicated devices may not be readily available. In such situations thesensors in the cell phones, which are readily available, may be used todetect and automatically transmit useful information. This sectiondescribes some of the applications that help in measuring certain vitalsigns of the human body. These measurements may not be very accurate andare not meant to replace the actual medical devices. The focus is on thefact that while actual medical help arrives for help, these devices canhelp the Para-medical people prepare for the medical emergency.

A continuous feed of patient's vital signs to the first responders andthe hospital is important. It is also important for the hospitalreceiving the patient to know the approximate time of arrival of thepatient. Wireless sensors and modern networks can help in communicatingthis information (UK Department of Health 2005). One of the advantagesof video streaming is live information sharing amongst several people.For example the video stream from the scene may be seen by thedispatcher and simultaneously seen by the first responders. A studyperformed concluded that the live video feed to responders and at thecommand center was very useful (SAMU France online study). There havealso been other proposed designs of multimedia based emergency services(Nena.org NG911 project online).

The current emergency Dispatch Protocols have been designed with theassumption that 911 calls are voice calls. The dispatchers have alimitation of not having direct access to the affected persons. Theyhave to rely a lot on the communication and interrogation skills toelicit optimum information from the caller. The Dispatch Protocols are adocument that consists of written, step by step, instructions for thedispatcher on how to gather information about the emergency and thenfollow it up with instructions on what to do for each situation. Thismay include giving comfort to the caller, if he or she is the affectedperson. It may also include giving some first aid advice andinstructions to the caller, before the emergency help arrives. Theseprotocols are almost like an algorithm for each situation. They continueto be improved and developed by the National Academy of EmergencyMedical Dispatchers. However, the time to gather information is limited,and even with these protocols, there may be situations where it takessome time to reach a decision. An optimum time to make a decision isconsidered to be 60 seconds, also called the “Sixty second dilemma.”This time was set arbitrarily, although a more reasonable time isexpected to be about 75-90 seconds. It is expected that with multi-mediacalls, the time limit of 60 seconds may become realistic and may even beless than that.

The next generation Dispatch Protocols would use several technologies tohelp reduce the time it takes to help and also improve the quality ofthe help provided. This is achieved by better diagnosis of the problemthat in turn can prepare the medical personnel to serve the patientbetter as he is on the way to the hospital. This reduction in time willbe possible by designing the protocols so that the operator asks fewerquestions. The number of instructions given to the caller is alsoreduced.

The following is an example of questions asked by the dispatcher for a911 call. The corresponding change in the protocol is also given at thesame time. The caller complains of chest pain and possible heartproblems for a patient. The first question that the dispatcher asks is“Is the patient alert?” A modified protocol would not need to ask thisquestion. The smart phone will have sensors to relay this informationdirectly to the dispatcher. Given a video call, the dispatcher couldalso observe the patient for himself. The most instructions a callerwould receive would be to move the cell phone camera to view thepatient. The follow up question under old protocol is to find out if thepatient is breathing normally. This question becomes redundant given thenew technology. There are several sources from which the dispatcher canmake this conclusion about breathing without having to ask the caller.The cell phone accelerometer can determine the breathing pattern. Anaudio transmission of breathing sounds can be used to conclude thequality of breathing. For example, a noisy breathing pattern may be dueto asthmatic problems.

According to the above example, the dispatcher will have input fromseveral sources for a given call. The display screens for the dispatchermay need to be modified. A proposed screen layout would allow the screento receive multiple video streams. These streams could be coming fromthe caller, the dispatch team, and potentially other sources. Similarlythe image windows on the screen can display simultaneously a patient'smedical history as sent from some remote location. In addition there isfacility to receive text and email messages. A part of the screen couldalso display a map of the location from where the 911 call is made.

Various methods of measuring and transmitting vital signs of a humanusing mobile phones are described. A mobile phone based blood pressureestimation technique is described herein that replaces the traditionalcuff and stethoscope traditionally used for blood pressure measurementwith a combination of the mobile phone's built-in microphone and camera.

In particular, a differential technique of estimating blood pressurewith the heart beat and pulse data obtained from two mobile phones isdescribed. The procedure consists of first synchronizing the time onboth the phones using a program such as Bluetooth. Then the heart beatsand pulse signals are obtained using the phones. The systolic anddiastolic pressure is determined by computing the pulse pressure and thestroke volume from the data recorded. By comparing the estimated bloodpressure values with those measured using a commercial blood pressuremeter, encouraging results of 95-100% accuracy were obtained.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows a schematic diagram of a multimedia based call scenarioutilizing Next Generation 9-1-1.

FIG. 2 shows plots of heart rate measurements taken from video analysis,both before and after filtering.

FIG. 3 shows a plot of respiration rate taken from accelerometer sensordata, indicating normal breathing.

FIG. 4 shows a plot of respiration rate taken from accelerometer sensordata, indicating irregular breathing.

FIG. 5 shows a plot of CPR compression rate taken from accelerometersensor data.

FIG. 6 shows a plot of displacement rate during CPR compressions takenfrom accelerometer sensor data.

FIG. 7 shows a general scheme for blood pressure estimation according tothe present disclosure;

FIG. 8 shows a general scheme for synchronization of two mobile phones;

FIG. 9 shows functional components for each mobile phone duringsynchronization;

FIG. 10 shows a flow of events during synchronization of two mobilephones;

FIG. 11 shows functional components in a client clock in a mobile phoneduring clock synchronization;

FIG. 12 shows a general scheme for deriving pulse signal from the fingertip and calculating heart rate;

FIG. 13 shows (a) a total window of data collected on pulse rate and (b)the data split into smaller time frames;

FIG. 14 shows a general scheme for sequential execution of events duringblood pressure estimation;

FIG. 15 shows an analysis of identified peaks in captured datapertaining to blood pressure;

FIG. 16 shows heart beat data recorded with a microphone of a mobilephone;

FIG. 17 shows (a) data collected for blood pressure estimation havingsharp peaks, and (b) data collected for blood pressure estimationshowing a pattern of flat peaks;

FIG. 18 shows a plot of systolic pressure values measured for a durationof 60 seconds in a single data set;

FIG. 19 shows a plot of systolic pressure versus VTT compared to datafor various individuals;

FIG. 20 shows the generalized structure of a data packet; and

FIG. 21 shows microphone frequency response for three different mobilephones.

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS

Generally, the present disclosure relates to improved 911 services andemergency dispatch protocols utilizing sensors and applicationsavailable on mobile phones. This disclosure also relates to measurementof vital signs of a human using mobile phones, including a method forestimating blood pressure utilizing a mobile phone equipped with sensorsand applications.

With the advent of new telecommunications technologies such as wirelessand the Internet, it has been made possible to use multimedia servicesfor 911 rather than strictly voice services for 911 calls. The newservices can use images, video and text transmissions in addition to thetraditional voice transmission. The architecture for these enhancedservices, called Next Generation 911 services (NG911), has been underdevelopment since 2005. The central point of interaction for 911 callsis the dispatcher at a 9-1-1 center who receives the emergency calls.The dispatcher has to respond to the emergency in an appropriate mannerwithin a very short time. This involves asking specific questions fromthe caller to infer the type of emergency and then to decide on a courseof action. Protocols provide a method to standardize this process acrossall 911 emergency centers. The operators handling the 911 calls usestandard protocols, called Dispatch Protocols, to answer the calls forhelp. These protocols guide the operator about what questions to ask andwhat actions to take during a given emergency. The existing DispatchProtocols assume a voice-only call for 9-1-1.

With the deployment of NG911, the Dispatch Protocols can be modified tomake use of the multimedia calls to make the process of answering 911calls more efficient. The new technology turns mobile phones intopersonal devices with several embedded sensors. In the presentdisclosure, the sensors in a mobile phone are used for collecting vitalsigns and transmitting them to the 911 dispatcher. For example, anaccelerometer, embedded in a cell phone, can be used to measure thebreathing rate of a person with about 90% accuracy, or it can be used toguide a person to give proper CPR in case of emergency. Similarly, thecell phone sensors can be used to get the readings of heart rate with anaccuracy of about 90%. This can further enhance the effectiveness of theDispatch Protocols by reducing the time to respond to an emergency.

The effectiveness of the improved dispatch protocols will be dependentupon several different factors. One of the most important measures ofthe success would be the reduction in time it takes for an operator todecide the level of seriousness of the incident and dispatch help.Another factor is the number of transactions involved. Many questionsthat an operator has to ask the caller during a voice call may becomeredundant in a multimedia call using video. So the number of questionseliminated is another measure of effectiveness. Another factor isbandwidth. Many of the vital signs may be measured by the cell phonesensors and can also be automatically transmitted. The amount of suchdata that can be automatically sent to the operator would be anothermeasure. The quality of decision making and quality of service at thedispatch center are factors that should also improve. This can bemeasured by several factors like a reduction in the number of un-neededdispatches, and an increase in the number of lives saved.

The cost factor is also important from the call center point of view.Currently the call centers are equipped for voice only calls. A setupthat allows multi-media calls would require substantial additionalinvestment for call centers. This would include equipment costs as wellas training costs.

A 911 call may be made by the affected people themselves. Such peoplemay be facing cognitive impairment or even physical impairment. Theoperator answering the call goes through several steps during the call.During a voice call, the operator is dependent on the caller to give anaccurate description of the problem and answer all the questionscorrectly. The operator may then have several instructions for thecaller to help him while the dispatch personnel arrive at the scene. Theoperator is again dependent on the caller to follow the instructions.The operator does not have an easy way to judge if the instructions arebeing followed accurately or if they are helping. But in the case of avideo call, the operator can follow the caller much more easily, can seethe scene of the incident, and can also see if the instructions arebeing followed.

In situations where the caller is a close relative or friend of theaffected person, the caller may have cognitive impairment. In thesesituations using a multimedia call, including the video would help theoperator make better decisions. The operator can actually see the sceneas he guides the caller while the dispatch team is on its way.

Power management may become an important issue during a multimedia phonecall. If the video camera is on during a 911 call, the power consumptionincreases substantially. A camera can use 25% of the CPU's capabilitieswhen the camera is being operated. It is important that the phone doesnot run out of battery, resulting in loss of connection. This implies aprudent use of the video camera or other features of the camera thatconsume power. Basically, the only way to ensure this is to use thevideo features for short duration as and when needed. The operator hasan important role to play in this. It is the operator who can guide thecaller about when to turn on the camera and when to turn it off. If thecamera can be remote controlled for these features then it makes iteasier for the operator to turn it on or off based on the need.

Some of the vital signs that can be recorded or otherwise monitored andrelayed to the dispatcher include heart rate, respiration rate,respiration difficulty, the effectiveness of CPR being administered, andother general activities that can be detected using video. Thedispatcher can also have the ability to remotely control the mobilephone being used in order to better isolate those features beingexamined.

Example 1 Heart Rate

Heart rate (“HR”) is a vital sign of human health. To measure HRrequires a trained person or device placed at specific place on thebody. Using a cell phone to measure HR may seem difficult, but there areapplications that have been developed that help in reasonably accuratemeasurement. One of these applications is based on the concept of PhotoPlethysmography (“PPG”). It uses the camera and the LED flash availableon most cell phones. The principle is that every heart beat results inpumping the blood through the blood vessels, including the capillariesat the finger tips. This results in variation in light intensity thatpasses through the finger tips. When a finger is placed on the cameralens of a cell phone and the LED flash its utilized, a video is capturedof the light passing through the finger tip. An analysis of the changesin the pattern of this light intensity is made, giving the heart ratewith an accuracy of at least 95% (NENA i3 Technical RequirementsDocument).

FIG. 2 shows the plot of results from video analysis for heart ratemeasurement, both the raw data and the filtered data. The heart rate ismeasured by calculating the number of peaks within a certain windowframe and then using the equation:

HR=n*60/W _(t)

Where n is the number of peaks in the video frame and W_(t) is thelength of the frame in seconds.

The results were calibrated using a commercial heart rate monitor. Inorder to take readings at various heart rates, the subject had to gothrough different intensities of exercise. For the purpose of analysisthe video frame was divided into smaller windows of 5 seconds. Table 2below shows these readings using this method as well as the readingsobtained from commercial monitor. The actual HR column shows the HeartRate measured with a commercial HE monitor. The next 2 columns show theHR values and its accuracy when the 5 second window is used. Finally,the table shows the HR values and accuracy when a 10 second window isused.

TABLE 2 Comparison of Heart Rate Measurements 5 second Window 10 secondWindow Actual HR value Accuracy Value Accuracy 102 108 94.11 102 100 10896 88.89 102 94.44 114 108 94.74 114 100 132 132 100 132 100 154 14493.51 150 97.4

Example 2 Respiration Rate and Difficulty

Respiration is another important vital sign. The accelerometer sensor ina mobile phone can be used to measure the respiration rate of a person.The cell phone is placed on the upper abdomen of a person to get thebest measurements for the respiration rate. It should be placed in ahorizontal position, with its top and bottom parallel to an imaginaryline running between the person's head and feet. The accelerometer onthe cell phone records the measurement of acceleration approximatelyevery 20 milliseconds. FIG. 3 shows a plot of the respiration asmeasured by the accelerometer of a cell phone. The Respiration rate canbe measured with an accuracy of about 98%.

Another important measure of breathing is the ease or difficulty ofbreathing. The accelerometer graph can also indicate if the breathing isuneven. The microphone (as well as accelerometer) can capture thebreathing sounds coming from the lungs if the cell phone is placed nearthe upper chest. This can indicate if the breathing is hard or there areany wheezing or guttural sounds coming from lungs during breathing. FIG.4 shows a pattern of irregular breathing. The pattern shows the personbreathing slow for certain period of time (about 15 times per minute)and then followed by fast breathing for a period of time (about 45 timesper minute).

Example 3 CPR Assistance

In case of medical emergencies where the patient has stopped breathingor has a heart problem, a timely and correct administration of CPR, withappropriate frequency of chest compressions, can mean the differencebetween life and death. CPR needs to be given as soon as possible. Mosttimes the people near the patient may not know how to give CPR. In suchcases the 911 operator gives verbal instructions over the phone to helpsomeone give CPR to the patient. While there are medical devices thatcan help administer proper CPR, these devices may not be available oraccessible in emergency situations at homes or public places. Here againa cell phone with an accelerometer can help in such situations. Anapplication can be utilized that measures the frequency of chestcompressions and the depth of chest compression. The application canprompt in case the person giving the CPR needs to change the frequencyor depth of compression. The phone can be placed directly on the chestor can be wrapped around the hand by a cloth and then placed on thechest. FIG. 5 shows a sample of accelerometer data plot while doing aCPR.

FIG. 6 shows a plot of displacement during administration of CPR, asmeasured by a smart phone accelerometer sensor. The plot shows a regularpattern of chest compression during CPR. The calculation of displacementis done by a two step procedure. The raw data from the accelerometergives the acceleration reading approximately every 20 milliseconds. Thefirst step is to find the velocity by using the equation:

V _(t) =V ₀ +V _(d),

where V₀ is the starting velocity and V_(d) is the change in velocityover a period of time and V_(t) is the velocity after time t. The secondstep is to calculate the displacement from velocity using the equation:

Displacement=(V ₀ +V _(t))/2*t,

where t is the time period.

The readings for these experiments were taken in a CPR class. Severalreadings were taken as the instructor performed the CPR on a Mannequin.Readings were also taken as the students performed the CPR on aMannequin. The number of students in a class varied from 5 to 7. Theexperiment was done at 3 different classes. The accuracy of the plots isindicated by the fact that the instructor must perform the CPR to adepth of about 2 inches.

Example 4 Remote Control of Sensors

On many occasions, callers are not able to operate the sensors and feedthe dispatcher information about the emergency situation. In time ofpanic, it is also difficult for the caller to make the rightobservations and answer the dispatcher correctly. Because of theintroduction of multimedia technology in NG911, the dispatcher should beable to extract the information about the scene by making observationsand measurements remotely and should be able to instruct the calleraccordingly. Table 3 below shows several ranges for different cell phonecamera features. The dispatcher can remotely take control of the camerafeatures in order to get better view of the emergency scene. This isuseful when a caller is physically disabled or is cognitively impairedto the point he or she cannot effectively follow the instructions.

TABLE 3 Camera Features Feature Range Zoom 1-40 Brightness 0-10 Contrast0-10 Sharpness 0-30 Saturation 0-10 Rotation  0-360

Cell phone cameras are available with several sensors embedded in them.Many of these sensors also have controls to change their settings. Forexample, video cameras in a cell phone will have a setting for zoomingor panning or changing the resolution of the pictures taken. Thesesettings can be changed by using control buttons on the cell phone.Normally, the 911 operator will have to instruct the caller to changethese settings, if needed. But new technology provides the ability forapplications to allow the 911 operator to take control of thesesettings. In these cases, the operator may choose to zoom in or pan outwhen he feels the need for it. Similarly, the audio controls of volumemay be controlled remotely if the operator is trying to listen to abreathing sound.

Example 5 Detection of Activity

Many times, the callers/informants are not sure about the patient'sstate, possibly due to panic or poor cognition. It has been observedthat every human activity has a direct impact on the change in bit rateassociated with the frame in a video stream. The bit rates associateswith various activities are shown below in Table 4. For example, in theabsence of a user in the view port indicating a completely static scene,the bit rate maintains a very low value of 4 Kbps for every resolution.However, there is a change in bit rate when the scene includes a personeven if the subject is completely at a standstill. The increase in thebit rate for different activities for the same resolution is due to themovement of exposed body parts involved in the activity. Hence, it canbe concluded that bit rate increases even with a small amount of motionexerted by the human body without the user's knowledge. These changesare evident when a person is breathing normally and breathing heavily.It should be possible to detect even small changes in the scene, such asheavy breathing, screaming, and other body movements (using audio orvideo). One can extract this information in real-time without anycomplex image/video processing. This can help the operator make aninformed decision about the state of the injured person. The dispatcherscan look at the state of the person even when no one is around.

TABLE 4 Activity Detection Activity KB/s No User 4.19 Eye Blink 24.57Smile/Scream 20.70 No Breathing 19.06 Normal Breathing 35.11 HeavyBreathing 62.39

Example 6 Modified Emergency Dispatch Protocol Questions

Table 5 below shows the typical list of emergency dispatch protocolquestions asked by the dispatcher to a caller during an emergency call,depending on the nature of the reported injury. The columns to the rightof the question indicate the cell phone sensor data that could be usedto help answer these questions, the instructions or requests that thedispatcher could give the caller, and the quantitative data that ismeasured.

TABLE 5 Current Protocol Modified Name/Question Cell phone sensorquestion/action in Protocol used/results requested Quantitative dataBleeding/ Laceration is the patient alert Video images of Move thecamera over 1. Change in pixels moving eyes, or the patient. Zoom orvalues arms/hands, or legs. change resolution (if 2. Zoom in, Imageanalysis is needed). local/remote (N done. times) 3. Resolution change,local/remote Is the patient 1. Accelerometer data Turn on the 1.Accelerometer breathing showing respiration accelerometer. Place datafor rate of normally? rate. the phone on the respiration 2. Microphoneaudio patient's upper 2. Volume control, to hear breathing abdomen.local/remote sounds for wheezing etc. Where is the Video images of Movecamera to 1. Change in Pixels bleeding from? moving eyes, or injury.Zoom in the values arms/hands, or legs. camera to the injury (if 2. Zoomin, Changes in pixels can not possible from local/remote (N lead toconclusions remote). times). about limb movements 3. Resolution change,local/remote Is blood squirting Video image of redundant out? bleedingBleeding - from Video image of Zoom in the camera 1. Change in Pixelswhere, How bleeding on the bleeding. values much, How long, 2. Zoom in,Can it be local/remote (N controlled with times). pressure 3. Resolutionchange, local/remote Can the patient Use of video and Take the phonenear 1. Volume Control, answer your audio enhances the the patient.Increase local/remote. questions? interaction the volume of themicrophone (if needed) Eye Problems/ Injuries Is the patient Videoimages of Scan the camera over 1. Change in pixels alert? moving eyes,or the patient. Zoom in or values. arms/hands, or legs. changeresolution (if 2. Zoom in, Image analysis is needed). local/remote (Ndone. times). 3. Resolution change, local/remote Is the patient 1.Aceelerometer data Turn on the 1. Accelerometer breathing showingrespiration accelerometer data for rate or normally? rate. remotely.Place the respiration 2. Microphone audio phone on the patient's 2.Volume control, to hear breathing upper abdomen. local/remote sounds forwheezing etc. What caused the Video images of the Focus the cameraover 1. Zoom in, injury? injury and the overall the patient's eyes.local/remote (N Chemicals scene would provide a Zoom camera over thetimes). Foreign object better diagnosis of the affected eye (if 2.Change Impaled object situation. needed). resolution, Direct blow Changethe resolution local/remote. Flying object (if needed). Welding/nearwelder Is eyeball cut Video image of the Zoom camera over the 1. Zoom inthe open or leaking eyes. The zoom affected eye (if camera, local/remotefluid? answers this question. needed). (N times). Change the resolution2. Change (if needed). resolution, local/remote. Are there any Videoimage of other Move the camera to 1. Zoom in the other injuries? injury,if any any other injury camera, local/remote (N times). 2. Changeresolution, local/remote. Fall Victim Is the patient Video images ofMove the camera over 1. Change in pixels alert? moving eyes, or thepatient. Zoom or values. arms/hands, or legs. change resolution (if 2.Zoom in, Image analysis is needed). local/remote (N done. times). 3.Resolution change, local/remote Is the patient 1. Aceelerometer dataTurn on the 1. Accelerometer breathing showing respirationaccelerometer. Place for rate or normally? rate. the phone on therespiration 2. Microphone audio patient's upper 2. Volume control, usedto hear breathing abdomen. local/remote sounds for wheezing etc. Whatkind of Video image of the Move the camera over 1. Zoom in the surfacedid the surface. the surface where camera, local/remote patient land on?patient fell. (N times). 2. Change resolution, local/remote. Are thereany Video image of the Move the camera over 1. Zoom in the obviousinjuries? injuries the injury. camera, local/remote What are they? (Ntimes) 2. Change resolution, local/remote Is the patient able Videoimages of Move the camera over 1. Change in pixels to move their movingeyes, or the hands and toes of values. fingers and toes? arms/hands, orlegs. the patient. 2. Zoom in, Ask him to move local/remote (N them.times). 3. Resolution change, local/remote Bleeding - from Video imagesof Zoom in the camera 1. Change in Pixels where, How moving eyes, or onthe bleeding. values much, How long, arms/hands, or legs. 2. Zoom in,Can it be local/remote (N controlled with times). pressure? 3.Resolution change, local/remote. Heat/Cold Exposure Is the patient Videoimages of Move the camera over 1. Change in pixels alert? moving eyes,or the patient. values. arms/hands, or legs. Zoom or change 2. Zoom in,Image analysis is resolution (if needed). local/remote (N done. times).3. Resolution change, local/remote Is the patient 1. Accelerometer dataTurn on the 1. Accelerometer breathing showing respirationaccelerometer. Place data for rate or normally? rate. the phone on therespiration 2. Microphone audio patient's upper 2. Volume control, tohear breathing abdomen. local/remote sounds for wheezing etc. Can thepatient Use of video and Take the phone near 1. Control Volume answeryour audio enhances the the patient. of the microphone questions?interaction Increase volume of the from local/remote. microphone ifneeded. If the patient is Video images of pain Move the camera over 1.Change in pixels complaining of area may help better the area of pain.values. pain, where? diagnosis Zoom over the pain 2. Zoom in, area, ifneeded. local/remote (N times). 3. Resolution change, local/remote Doesthe patient Video/audio Take the phone to the 1. Control Volume respondto you interaction. Operator patient. of the microphone and followsimple interacts with the Change Microphone local/remote. commands?patient. Volume, If needed. Is the patient Video image of the Move thecamera over 1. Change in pixels sweating patient and image the patient'sface. values. profusely? analysis helps. Zoom in, if needed 2. Zoom in,Change resolution, if local/remote (N needed. times). 3. Resolutionchange, local/remote Is the patient Video images and data Operator asksthe 1. Pressure change dizzy, weak or from pressure sensor patient topress on the from touch screen. feeling faint? on screen. camera touchscreen to measure pressure. Industrial Accidents Is the patient Videoimages of Move the camera over 1. Change in pixels alert? moving eyes,or the patient. values. arms/hands, or legs. Zoom or change 2. Zoom in,Image analysis is resolution (if needed). local/remote (N done. times).3. Resolution change, local/remote Is the patient 1. Accelerometer dataTurn on the 1. Accelerometer breathing showing respirationaccelerometer. data for rate or normally? rate. Place the phone on therespiration 2. Microphone audio patient's upper 2. Volume control, tohear breathing abdomen. local/remote sounds for wheezing etc. Is thepatient able Video images of Move the camera over 1. Change in pixels tomove their moving eyes, or the hands and toes of values. fingers andtoes? arms/hands, or legs. the patient. 2. Zoom in, Ask him to movelocal/remote (N them. times). 3. Resolution change, local/remoteBleeding - from Video images of Zoom in the camera 1. Change in Pixelswhere, How moving eyes, or on the bleeding. values. much, How long,arms/hands, or legs. 2. Zoom in, Can it be local/remote (N controlledwith times). pressure? 3. Resolution change, local/remote. Stabbing/gunshot assault Is the patient Video images of Move the camera over 1.Change in pixels alert? moving eyes, or the patient. Zoom or values.arms/hands, or legs. change resolution (if 2. Zoom in, Image analysis isneeded). local/remote (N done. times). 3. Resolution change,local/remote Is the patient 1. Accelerometer data Turn on the 1.Accelerometer breathing showing respiration accelerometer. Place datafor rate or normally? rate. the phone on the respiration 2. Microphoneaudio patient's upper 2. Volume control, to hear breathing abdomen.local/remote sounds for wheezing etc. Is there more than Video images ofthe Move the camera over 1. Pan the camera one person scene the entirescene. local/remote injured? Pan the camera, if needed Is there morethan Video images of the Move the camera over 1. Zoom in one wound? Whatinjuries all the injuries. local/remote (N part(s) of the Zoom thecamera, if times). body is/are needed injured? Bleeding - from Videoimages of Zoom in the camera 1. Change in Pixels where, How moving eyes,or on the bleeding. values. much, How long, arms/hands, or legs. 2. Zoomin, Can it be local/remote (N controlled with times). pressure? 3.Resolution change, local/remote. Vehicular related injuries Is thepatient Video images of Move the camera over 1. Change in pixels alert?moving eyes, or the patient. Zoom or values. arms/hands, or legs. changeresolution (if 2. Zoom in, Image analysis is needed). local/remote (Ndone. times). 3. Resolution change, local/remote Is the patient 1.Accelerometer data Turn on the 1. Accelerometer breathing showingrespiration accelerometer. Place data for rate or normally? rate. thephone on the respiration 2. Microphone audio patient's upper 2. Volumecontrol, to hear breathing abdomen. local/remote sounds for wheezingetc. Are there any Video image of the Move the camera over 1. Pan thecamera hazards present? scene the entire scene of the (Is the sceneaccident. safe), Fire. Water, Pan the camera over HazMat, Wires theentire scene. down How many Video image of the How many patientspatients are scene. are injured? Are all of injured? Are all thepatients free of the of the patients vehicle? free of the vehicle? Whattypes of Video image of the Move the camera over vehicle(s) are scene,focus on the the vehicle involved? vehicle Is anyone trapped Video imageof the Is anyone trapped in in the vehicle? scene, focus on the thevehicle? injured Traumatic Injury Is patient alert? Video images of Movethe camera over 1. Change in pixels moving eyes, or the patient. Zoom orvalues. arms/hands, or legs. change resolution (if 2. Zoom in. Imageanalysis is needed). local/remote (N done. times). 3. Resolution change,local/remote Is patient 1. Accelerometer data Turn on the 1.Accelerometer breathing showing respiration accelerometer. Place datafor rate or normally? rate. the phone on the respiration 2. Microphoneaudio patient's upper 2. Volume control, to hear breathing abdomen.local/remote sounds for wheezing etc. Bleeding - from Video images ofZoom in the camera 1. Change in Pixels where, How moving eyes, or on thebleeding. values. much, How long, arms/hands, or legs. 2. Zoom in, Canit be local/remote (N controlled? with times). pressure? 3. Resolutionchange, local/remote. Where is the Video images of the Move the cameraover 1. Zoom in, patient injured? injury the injury local/remote (Ntimes). Abdominal Pain Is the patient Video images of Move the cameraover 1. Change in pixels alert? moving eyes, or the patient. Zoom orvalues. arms/hands, or legs. change resolution (if 2. Zoom in, Imageanalysis is needed). local/remote (N done. times). 3. Resolution change,local/remote Is the patient 1. Accelerometer data Turn on the 1.Accelerometer breathing showing respiration accelerometer. Place datafor rate or normally? rate. the phone on the respiration 2. Microphoneaudio patient's upper 2. Volume control, to hear breathing abdomen.local/remote sounds for wheezing etc. How does the Make the person sitMove the camera patient act when and observe on video showing thepatient. he/she sits up? Ask him to sit Has the Patient Video images ofvomit Has the patient 1. Zoom in, Vomited? If yes, can help. Number ofVomited? If yes, local/remote (N what does vomit pixels and color Movethe camera over times) look like? changes of pixels may the vomit. 2.Change in lead to automatic Resolution, conclusions. local/remote.Allergies Stings Is the patient Video images of Move the camera over 1.Change in pixels alert? moving eyes, or the patient. Zoom or values.arms/hands, or legs. change resolution (if 2. Zoom in, Image analysis isneeded). local/remote (N done. times). 3. Resolution change,local/remote Is the patient 1. Accelerometer data Turn on the 1.Accelerometer breathing showing respiration accelerometer. Place datafor rate or normally? rate. the phone on the respiration 2. Microphoneaudio patient's upper 2. Volume control, to hear breathing abdomen.local/remote sounds for wheezing etc. How does the Make the person sitMove the camera patient act when and observe on video showing thepatient. he/she sits up? Ask him to sit. Does the patient Video image ofbite Move the camera over 1. Zoom in, have any rashes area. Increasedthe rash area of his/her local/remote (N or hives? resolution and pixelbody. Increase the times). analysis would show resolution, if needed. 2.Change in the seriousness of the Zoom the camera resolution rash overthe rash, if local/remote. needed Back Pain Is the patient Video imagesof Move the camera over 1. Change in pixels alert? moving eyes, or thepatient. Zoom or values. arms/hands, or legs. change resolution (if 2.Zoom in, Image analysis is needed). local/remote (N done. times). 3.Resolution change, local/remote Is the patient 1. Accelerometer dataTurn on the 1. Accelerometer breathing showing respirationaccelerometer. Place data for rate or normally? rate. the phone on therespiration 2. Microphone audio patient's upper 2. Volume control, tohear breathing abdomen. local/remote sounds for wheezing etc. BreathingProblems Is the patient Video images of Move the camera over 1. Changein pixels alert? moving eyes, or the patient. Zoom or values.arms/hands, or legs. change resolution (if 2. Zoom in, Image analysis isneeded). local/remote (N done. times). 3. Resolution change,local/remote Is the patient 1. Accelerometer data Turn on the 1.Accelerometer breathing showing respiration accelerometer. Place datafor rate or normally? rate. the phone on the respiration 2. Microphoneaudio patient's upper 2. Volume control, to hear breathing abdomen.local/remote sounds for wheezing etc. Is the patient able Video/AudioTake the phone to the 1. Volume control to speak in full interactionwith patient. local/remote. sentences? patient Increase volume of themicrophone if needed. Is the patient Video images of Move the cameraover 1. Zoom in, drooling of person's face. Pixel the patient's mouth.local/remote (N having a hard changes would lead to Zoom in the camera,if times). time swallowing? automatic conclusions needed 2. Change inabout drooling and Increase resolution, if resolution, swallowingproblems. needed. local/remote Chest Pain/Heart Problems Is the patientVideo images of Move the camera over 1. Change in pixels alert? movingeyes, or the patient. Zoom or values. arms/hands, or legs. changeresolution (if 2. Zoom in, Image analysis is needed). local/remote (Ndone. times). 3. Resolution change, local/remote Is the patient 1.Accelerometer data Turn on the 1. Accelerometer breathing showingrespiration accelerometer. Place data for rate or normally? rate. thephone on the respiration 2. Microphone audio patient's upper 2. Volumecontrol, to hear breathing abdomen. local/remote sounds for wheezingetc. Is the patient Video images of the Move the camera over 1. Changein pixels nauseated or person. Image analysis the patient's face.values. vomiting? is done Zoom in, if needed. 2. Zoom in, Is the patientIncrease the local/remote (N sweating resolution, if needed. times).profusely? 3. Resolution change, local/remote Is the patient Use cellphone sensors - Turn on the 1. Accelerometer experiencing accelerometer,accelerometer. data rapid heart rate video, audio to Place the camera on2. change in Pixel with chest pain? determine heart rate the patient'schest. intensity Ask the patient to put finger on the camera lens.Diabetic Problems Is the patient Video images of Move the camera over 1.Change in pixels alert? moving eyes, or the patient. Zoom or values.arms/hands, or legs, change resolution (if 2. Zoom in, Image analysis isneeded). local/remote (N done. times). 3. Resolution change,local/remote Is the patient 1. Accelerometer data Turn on the 1.Accelerometer breathing showing respiration accelerometer. Place datafor rate or normally? rate. the phone on the respiration 2. Microphoneaudio patient's upper 2. Volume control, to hear breathing abdomen.local/remote sounds for wheezing etc. Is the patient Video image of theMove the camera over 1. Change in pixels sweating patient and image thepatient's face. values. profusely? analysis helps. Zoom in, if needed 2.Zoom in, Change resolution, if local/remote (N needed. times). 3.Resolution change, local/remote Headache Is the patient Video images ofMove the camera over 1. Change in pixels alert? moving eyes, or thepatient. Zoom or values. arms/hands, or legs. change resolution (if 2.Zoom in, Image analysis is needed). local/remote (N done. times). 3.Resolution change, local/remote Is the patient 1. Accelerometer dataTurn on the 1. Accelerometer breathing showing respirationaccelerometer. Place data for rate or normally? rate. the phone on therespiration 2. Microphone audio patient's upper 2. Volume control, tohear breathing abdomen. local/remote sounds for wheezing OD/Poisonings/Ingestion Is the patient Video images of Move the camera over 1. Changein pixels alert? moving eyes, or the patient. Zoom or values.arms/hands, or legs. change resolution (if 2. Zoom in, Image analysis isneeded). local/remote (N done. times). 3. Resolution change,local/remote Is the patient 1. Accelerometer data Turn on the 1.Accelerometer breathing showing respiration accelerometer. Place datafor rate or normally? rate. the phone on the respiration 2. Microphoneaudio patient's upper 2. Volume control, to hear breathing abdomen.local/remote sounds for wheezing etc. Is the patient Video images of theMove the camera over 1. Zoom in, having difficulty person. Number of thepatient's mouth. local/remote (N swallowing? pixel changes and the Zoomin the camera, if times). color changes of needed 2. Change in pixelswould lead to Increase resolution, if resolution, automatic conclusionneeded. local/remote about swallowing problems. Psychiatric/ BehavioralProblems Is the patient Video images of Move the camera over 1. Changein pixels alert? moving eyes, or the patient. Zoom or values.arms/hands, or legs. change resolution (if 2. Zoom in, Image analysis isneeded). local/remote (N done. times). 3. Resolution change,local/remote Is the patient 1. Accelerometer data Turn on the 1.Accelerometer breathing showing respiration accelerometer. Place datafor rate or normally? rate. the phone on the respiration 2. Microphoneaudio patient's upper 2. Volume control, to hear breathing abdomen.local/remote sounds for wheezing etc. Can the patient Use of video andTake the phone to the 1. Control Volume, answer your audio enhances thepatient. Increase the local/remote. questions? interaction volume of themicrophone (if needed)

Example 7 Cuffless Differential Estimation of Blood Pressure

The differential estimation of blood pressure involves the use of twomobile phones. One is for recording the heart sounds using the built-inmicrophone and the other is for recording the video of the pulse dataobtained from the subject's finger.

This technique consists of three phases. First is the synchronizing oftime between the two mobile phones, preferably using Bluetooth, followedby the second phase which involves locating an appropriate spot on thechest for recording heart sounds. The audio data of the heart sound isused to calculate the heart rate. The third phase involves calculationof systolic pressure, pulse pressure and the diastolic pressure. FIG. 7shows the overall procedure of the differential estimation technique.Pulse signals are obtained using the video recording application whenthe finger is placed over the lens and heart sounds are recordedsimultaneously using the audio application by placing the other phoneover the chest as shown in FIG. 1.

The process starts with synchronization of time between two mobilephones, preferably using Bluetooth. After synchronization, the camera onone mobile phone records the pulse in the finger at 24 fps via a videorecording application. An LED flash lights up when the applicationstarts and then the start-time of video recording is saved in thephone's SD card. Similarly, the second mobile starts recording the heartsounds via the audio recording application with the phone's microphoneheld close to the chest. Care has to be taken that the microphone'sopening is held tightly to the skin over the chest to avoid recordingexternal noise.

For synchronizing the clocks on both the mobile phones, asynchronization protocol was developed. This protocol is similar to thePrecision Time protocol-IEEE 1588 used in wired networks. Thesynchronization procedure preferably utilizes Bluetooth and follows aMaster-Slave architecture. The mobile phone which receives thesynchronization messages acts as the master and the one sending acts asthe slave. FIG. 8 shows the synchronization mechanism between the twophones. In this example, Nexus One and HTC Hero mobile phones were usedrunning Android operating system. Any suitable phones with the necessarycapabilities using any suitable operating system can be utilized. TheNexus One was equipped with a Broadcom BCM4329 chipset supportingBluetooth 2.1+EDR (Extended Data Rate technology) along with 802.11nWiFi. The Hero supports Bluetooth 2.1+EDR along with a 802.11b/g WiFiconnectivity. The phones communicated via the Android BlueZ, a Bluetooth2.1 compatible stack capable of running on any Bluetooth chipset. TheNexus One was used for video recording and acted as the master. The Herowas used for heart beat recording and acted as the slave. Thesynchronization procedure consists of two steps.

First is the master-slave message exchange mechanism. The layeredarchitecture comprises of three important layers namely Application,Bluetooth Stack and the Communication layers. FIG. 9 shows thefunctional components described for each layer. The master device'sapplication layer consists of a message relay module to relay a replymessage, whereas the slave's application layer consists of message relayand time synchronization modules to calculate the exact time to be seton the host device relative to the master device. The Bluetooth stackand the communication layers are the same for both the devices.

Second is the slave device synchronization to master clock. The masterdevice's timestamp is obtained through Bluetooth and is fixed on theslave device. This simple method synchronizes time on two devices.Differential estimation of blood pressure requires time calculationsprecise to milliseconds. Ignoring transmission delays will lead todegradation in the system's accuracy. Hence, transmission delays have tobe calculated during the procedure. This requires the timestamps of boththe systems. FIG. 10 depicts the working of the synchronization process.The first step is the standard establishment of a Bluetooth connectionafter pairing with the device. Once the connection is established, theslave device sends a timestamp to the master and stores it as Ts. Assoon as the master device receives the timestamp, it sends back its owntimestamp to the slave device. This returned timestamp is recorded asT_(R) along with the timestamp when T7 was received by the slave deviceas TOS. The process is repeated for 30 seconds and the timestamps arerecorded. This process is followed by the client clock synchronizationmodule in the slave device. This module has four major operations:estimating the roundtrip time, estimating the offset, calculating themaster time, and setting the slave clock. FIG. 11 shows the componentsinside a client clock synchronization.

The roundtrip time is given by the following equation:

RTT=T _(S) ′−T _(S)  (1)

The one-way transit time between the terminals is one half of theroundtrip time given by the following equation:

$\begin{matrix}{T_{t} = \frac{RTT}{2}} & (2)\end{matrix}$

The offset between the two devices O_(t) is given by difference intimestamp of the master and slave device according to the followingequation:

O _(t) =T _(S)−(T _(R) −T _(t))  (3)

Subtracting the one-way transit time from the receiving time in themaster device will produce the accurate time difference between theslave and master device. The master clock time is estimated bysubtracting the offset with the slave clock.

After synchronization, the next step is localization of the heart beatand pulse. Localizing a heart beat is a challenging task, but necessaryfor accurately determining blood pressure. A brief description of heartsounds is as follows. Heart sounds are produced with the opening andclosure of heart valves. The heart produces mainly 4 types of sounds inone heart cycle denoted as S1, S2, S3 and S4. The first heart sound (S1−lub) is produced by the atrioventricular valves (i.e., mitral andtricuspid), and the second heart sound (S2, dub) is produced by thesemilunar valves (i.e., aortic and pulmonary valve). The third andfourth heart sounds are produced only in some rare conditions due togallop. In this method, only the first and second heart sounds need tobe recorded. Experiments were conducted on the chest region to find thebest spot for obtaining recordable heart sounds. The four locations ofthe valves were identified and selected for recording purposes. Based onthe experiments, the heart sounds captured from the microphone wereheard the loudest in the pulmonary region containing the pulmonaryvalve. Since the pulmonary valve is associated with the S2 sound, inmost cases the decibel level of S2 is higher than that of the S1. Table6 below gives the location of valves in the chest with respect to theintercostal space and sternum.

TABLE 6 Location of Valve on Chest Pulmonary valve second intercostalspace left upper sternal border Aortic valve second intercostal spaceright upper sternal border Mitral valve fifth intercostal space medialto left mid- clavicular line Tricuspid valve fourth intercostal spacelower left sternal border

The third step is pulse detection and heart rate calculation. Theprocedure for detecting a pulse works on the principle that every heartbeat pertains to a rush of blood in the blood vessels, even in thecapillaries at the finger-tips (Banitsas 2009). By placing the fingerover the camera and turning on the flash through a video recordingapplication, the following observations were made. During a systolicpulse when the capillaries are rich in blood, more light was absorbed bythe blood, leading to a low reflective index and darker frameintensities. Likewise, during a diastolic pulse, most of the light wasreflected leading to bright frames. The change in intensity of lightpassed through the finger creating an alternative pattern of waves.These changes in intensity with time were used to obtain the pulse of aperson.

FIG. 12 depicts the procedure for deriving the pulse signal from thefinger and calculating the heart rate. The pixel information from thevideo was split into individual Red, Blue and Green components. In mostof the frames observed, the prominent color was red. Every frame of thevideo was split into 4 quadrants and only the first quadrant wasconsidered for analysis because the fluctuations were more predominantin that region. Then the frames were split on a fixed length window (Wt)to check for the number of peaks(n) occurring at equal intervals of timeas shown in FIG. 13. FIG. 13(a) shows the total window of data, whileFIG. 13(b) shows the data split into smaller time frames. The heart rate(“HR”) was calculated by using the following equation:

FIG. 14 illustrates the sequential execution of events in the estimationprocess. The estimation process starts by first synchronizing the systemclocks of both the mobile phones. After this, each mobile startsrecording the appropriate data. In the example, the Nexus One recordedvideo at a rate of 24 fps and the HTC Hero recorded audio at a samplingrate of 8 KHz. Both the mobile phones should start recording within 10seconds after clock synchronization. Otherwise, letting the applicationto run for a long time creates a drift in the system clock as theprocessor is responsible for maintaining the system time. Moreover, whenthe CPU is loaded with high-priority processes, the processor cannotmaintain the clock's accuracy.

In the example, all the measurements were taken for a duration of oneminute with the video starting roughly five seconds after the audio. Thedata collected was processed offline. The first step was the denoisingof data and resampling. Audio data collected from the mobile phone waspassed through a 15th-order low-pass butterworth filter which allowsfrequencies only between 10-250 Hz since most heart sounds recorded arepresent within the 10-250 Hz frequency band. For computationaleffectiveness, the filtered audio was resampled to a lower rate of 1KHz. Similarly, the video frames were processed as explained above togive the value of red intensity.

The next step was determination of peaks from the data and identifyingappropriate peaks in the video corresponding to the audio signal. In allthe experiments, the recording process on both the phones did not startat the same time. In other words, both the audio and video data were notsynchronous with the starting time. To identify peaks, the waveforms hadto be aligned. To achieve this, the starting times of the recording werestored on the SD card of the two phones respectively as time stamps forcalibration. For instance, suppose the starting time of the audio datawas SS and the starting time of the video data was SV. The difference inthe time stamps will give the amount of time one data was ahead/laggingthe other. Algorithm 1 below explains the procedure followed incalibrating the two devices. Since the sampling frequency for audio was1 KHz, adding and deleting data points was easier and more precise inaudio than video at 24 fps. FIG. 15 illustrates the aligned data and howthe peaks are identified. In FIG. 15, sys represents the systolic peakof the pulse 45 and dia represents the diastolic peak of the pulse. S1and S2 are the first 46 and second heart sounds respectively. VTT isidentified by time difference between the dotted lines.

Algorithm 1 Calibration for Synchronous start of Data INPUT:Timestampsof Audio and Video Start (S_(S) and S_(V) ) OUTPUT:Synchronous audiodata SyncStart = S_(S) − S_(V) if SyncStart ≧ 0 then Add Null data forSyncStart milliseconds towards start of the audio data set end if ifSyncStart < 0 then Delete data for SyncStart milliseconds from the startof the audio data set end if

For systolic pressure, the time of arrival of S1 and the time of arrivalof the corresponding systolic peak are noted. The difference in thesetwo time of arrivals yielded a parameter called Vascular Transit Time(VTT). VTT is defined as the dotted lines, or time taken by the blood totravel from the heart to an extremity of the body for one stroke of theheart. The change in systolic pressure can be derived from the change inVTT with respect to a reference value as shown in the equation below(Foo et al. 2006). Hence the reason for calculating VTT.

ΔP _(s)=−0.425×ΔVTT  (5)

Based on Equation 5, the systolic pressure values corresponding to theVTT can be generated. Thus, the systolic pressure is given by thefollowing equation:

P _(s)=−0.425×VTT+214  (6)

The pulse pressure (Pp) and the stroke volume (SV) were computed asfollows. For stroke volume, the following equation was used:

SV(mL)=−6.6+0.25×(ET−35)−0.62×HR+40.4×BSA−0.51×Age  (7)

where ET(ms) is the ejection time and BSA is the Body Surface Area givenby the following equation:

BSA=0.007184×Weight^(0.425)×Height^(0.725)

Ejection time is defined as the time of ejection of blood from the leftventricle beginning with the opening of the aortic valve and ending withclosing of the aortic valve. The ejection time was estimated graphicallyfrom the heart sound recorded as shown in FIG. 16. The identification ofthe two sounds, namely S accompanied by opening of the aortic value andS2 accompanied by closing of the aortic value was explained above. Asshown in FIG. 16, the time difference between S and S2 sounds recordedusing the microphone is taken as the ejection time.

With the computed stroke volume, the pulse pressure was calculated inunits of mmHg using the following equation:

$\begin{matrix}{P_{p} = \frac{SV}{( {{0.013 \times {Wt}} - {0.007 \times {age}} - {0.004 \times {HR}}} ) + 1.307}} & (8)\end{matrix}$

Having obtained P_(s) and P_(p), the diastolic pressure (P_(d)) wascalculated from the following equations:

$\begin{matrix}{P_{s} = {P_{m} + {\frac{2}{3}P_{p}}}} & (9) \\{P_{d} = {P_{m} - \frac{P_{p}}{3}}} & (10)\end{matrix}$

Subtracting Equation 10 from 9 and rearranging, the diastolic pressureis calculated from the following equation:

The accuracy of the results relies entirely on the effective estimationof VTT from the graphs. Identification of the time instance of S1 soundfrom the audio was an easy task with proper filtering of the signal, butthe selection of the corresponding peaks in the video intensity wasdifficult in some situations due to presence of flat peaks instead of asharp peak for a systolic pulse in the video intensity plot. The flatpeak pattern appears due to the sampling error in the video. Thiscreated some confusion in the selection of a data point. FIG. 17(a)shows a plot with sharp peaks and 17(b) shows a pattern of flat peaks. Asample data set was considered and the VTT was obtained using both sharpand the flat peaked data points. Table 7 below shows the values of VTTand P_(s) computed from a single sharp peak data point and from a seriesof points present in a flat peak as two columns respectively. Forverifying the P_(s) estimated using this method, the blood pressure wasmeasured as 113 for a single subject using a commercial meter. FromTable 7, it appears that using a sharp peak, the VTT varied within asmall range and the P_(s) estimated was closer to the measured bloodpressure, whereas, with flat peaks there was a broad range of variationin the VTT and a considerable difference between the estimated and themeasured values. The median systolic pressure was 112 for the selectionof sharp peaks and 116 for flat peaks. Hence for an accurate measurementof pressure, the selection of data points plays an important role.

TABLE 7 Sharp peak Flat peak VTT Ps(mmHg) VTT Ps(mmHg) 240 112 227 118238 113 227 118 244 110 272 98 242 111 240 112 247 109 248 109 241 112222 120 235 114 228 117 245 110 234 115 245 110 230 116 240 112 229 117

Table 8 gives the accuracy of results upon selection of a sharp peakeddata. The measurements were collected from five individuals multipletimes. This data was to used to estimate the blood pressure using theproposed method. The systolic pressure was also measured using a bloodpressure meter. The accuracy is defined according to the follow ingequation:

$\begin{matrix}{{Accuracy} = \frac{{{Md}( P_{s} )} - {{Es}( P_{s} )}}{{Md}( P_{s} )}} & (12)\end{matrix}$

In the equation, measured (Md) P_(s) is the pressure measured using acommercial blood pressure meter and Estimated (Es) P_(s) is the pressureestimated as explained above. In Table 8, it can be seen that accuracyvalues varied between 90-100%. Also, the current method yielded anaccuracy above 95% in 85% of the data sets and of 90-95% in 15% of thedata sets analyzed.

TABLE 8 Md Es Accuracy Md Es Accuracy Ps Ps % Ps Ps % 110 110 100 97.4124 124 100 95.5 115 112 99.1 111 116 93.8 108 107 94.2 113 120 91.8 104110 98.3 110 119 99.1 118 116 95.0 112 113 96.5 121 115 96.4 114 11099.1 112 116 97.3 112 113 93.3 110 113 97 104

Accuracy of diastolic pressure is defined similar to Equation 12. Table9 below shows the accuracy for diastolic pressure calculated. Thediastolic pressure is approximated based on the amount of blood in theperson for his age, calculated from ET. The accuracy may change in thecase of blood loss or deficiency of blood due to abnormality.

TABLE 9 Md Es Accuracy Md Es Accuracy Pd Pd % Pd Pd % 64 68 93.75 70 7395.7 61 66 91.8 65 70 92.3 54 59 90.7 73 78 93.2 83 82 98.8 63 67 93.781 83 97.5 61 68 88.5

The vascular transit time (VTT) was analyzed based on the audio andvideo samples measured together. For every single pulse in the data set,a VTT can be derived. However, the VTT can change on ever pulse sincethere is a continuous variation of pressure in a human throughout theday. By using the proposed method, the systolic pressure was determinedwith a single VTT value but the probability of erroneous results washigh. Thus, for all the analyzed data sets, the median of the VTT wastaken to compute the blood pressure. FIG. 18 shows a plot for systolicpressure values for a duration of 60 seconds in a single data set. Thesample data set varied within a range of 24 mmHg. The median of thepressure value resulted in 113 mmHg which was exactly the same from theblood pressure meter for that instant.

The results from Table 10 below introduce the problem involved in theselection of number of data points. Table 10 shows the variation ofestimated (Es) systolic pressure (P_(s)) from measured (Md) basedselection of number of data points from a data set. Although thesystolic pressure can be estimated by selecting a single data point, theaccuracy of the values will be very low. Table 10 contains four datasets showing the pressure values with different number of data pointsselected. A single point was taken at random from minute a data set andcompared against the actual pressure value. In a similar manner, thepressure value was computed by selecting five and ten points. All theplots showed results close to actual pressure, but the closenessincreased with increase in the number of data points selected. From theTable 10, it can be inferred that the selection of at least 10 datapoints from the data set yielded closer values of pressure.

TABLE 10 Md 1 5 10 Sample Ps point points points 1 110 105 112 110 2 115110 110 112 3 108 101 112 107 4 104 115 114 111

The cuff-less differential blood pressure estimation technique describedherein uses mobile phone and their built-in sensors. Synchronization isfollowed by procedures for localizing and detecting heart beat and pulsesignals. The mathematical formulations used to estimate the systolic anddiastolic pressure produce accuracies between (95-100%).

Some technical and social issues arise which require consideration inthe use of the present cuffless differential estimation of bloodpressure. First, the method of measuring pulses with the video intensityis native without much filtering. Hence, the appearance of sharp peaksmay hardly be detected. The addition of special hardware may effectivelyincrease the accuracy and reliability of the device, but it voids thesystem's purpose. Also, the hardware specifications for all devices arenot standard. Device incompatibilities will most likely occur. Forexample, the LED flash available in one phone may not be present in alldevices as standard. Also, software version differences make theapplications to work only on certain devices. Heart beat recording alsorequires placement of the microphone over the bare chest. Takingmeasurements with the shirts on does not give desirable results. Hence,the measurement may not be possible in public. Further, people withchronic cardiac and vascular disorder may not follow the sameVTT-Systolic pressure relation. In addition, the accuracy of estimationtechnique was less when the tests were performed with the subject lyingflat on the ground. The subject preferably needs to be sitting uprightwith the arm holding the camera in level to the heart. Finally, thepresent investigation required the use of rooted phones which arepopular only among developers. API changes have to be made, toincorporate changes in these phone's system or the application should beadded as a core system process to be used by all public users.

Example 7A Sensitivity Analysis

The cuffless estimation technique's sensitivity was determined bystudying the relation between different parameters. Firstly, therelation between the cardiac output and a person's height and weight wasstudied. Cardiac output is defined as the volume of blood being pumpedby the heart in a time interval of one minute. Mathematically, it is theproduct of heart rate and stroke volume which were obtained as explainedabove. Because cardiac output increases with an increase in anindividual's body-surface area, the person's blood pressure alsoincreases linearly. Also, the cardiac output increases more rapidly withheight rather than with the weight of the individual, i.e., tallindividuals have higher cardiac output than obese individuals.

The sensitivity of the recording device also must be considered whileconsidering in the sensitivity analysis. The software cap on the Androidplatform for Nexus One prevents the hardware from performing to itsoriginal capability i.e., the hardware camera is reported to recordvideo at 720 pixels at a rate of 30 fps, but the recording capabilityhas been restricted to 24 fps at 480 pixels resolution. All thespecifications discussed here are results of testing the Nexus One'scamera under bright lighting conditions. When the picture gets darker,the frame rate decreases considerably. It was observed during the testsetup that the average frame rate dropped from 23 fps to 17-16 fps.Every frame occurred at an average time interval of 62.5 ms. Based onthis estimate, we conclude that the accuracy of ejection time variedbetween 0-120 ms in a data set and that the accuracy of results variedfrom 0-45 mmHg of mercury. Even the accuracy of pulse pressure rangedbetween 0-18 mmHg. For example, if a median error of +24 ms of ET isconsidered for the results, an actual 120/80 mmHg systolic pressure willshow as 145/96. To eliminate this kind of error phenomenon, the medianvalue of the ejection times was taken in a data set, thereby improvingthe current system's accuracy.

An analysis was made into whether factors such as age, height, andweight affect the system's sensitivity. The tests were performed withsubjects of varying age, height, and weight as listed in Table 11 below.

TABLE 11 Subject Age Height(ft) Weight(lbs) Data 1 22 6.0 149 Data 2 286.1 142 Data 3 23 5.8 140 Data 4 25 5.8 145 Data 5 13 5.5 123

By obtaining the VTT and P_(s) for the five subjects as explained above,the estimated values of Ps from VTT were compared with the pressuremeasured using a meter. FIG. 19 shows a plot of Systolic pressure vsVTT. The meter based readings are joined by a straight dark line in thecenter. The points labeled as ‘Data’ refer to different individuals. Theresults obtained by applying Equation 6 to all the individualsregardless of the age, height or weight for P_(s) computation, yieldedaccuracies of above 95%. Based on these results, it can be inferred thatfactors such as age, height, and weight did not impact the end result ofproposed system. It was also observed that, for an increase of 10 ms ofVTT, the systolic pressure dropped linearly by 4.25 mmHg. Based on theinformation from the sensitivity analysis, it can be concluded that thesystem will work under most conditions regardless of the physique of theindividual. However, the method has not been tested with people having ahistory of cardiac disorders. The results and accuracy could also varyfor people with ailments and disabilities due to changes in theirvascular system.

Example 7B Mobile Hardware Performance Analysis

The accuracy of results depends on the performance of a system under anyconditions. A performance analysis could shed further light on themodules within the system that may need improvement or adjustment forbetter estimation. The synchronization process and some hardwareconstraints are emphasized.

First, synchronization is a very important module in the proposeddifferential estimation of blood pressure. The proposed system uses twomobile phones in a master-slave relationship for collecting data. Whenthe time difference between the clocks increases, the probability ofselecting the correct S 1I sound for the corresponding video of thesystolic pulse decreases. If the clocks are working off sync, theresulting values from the estimation are incorrect.

To estimate the performance of the device, it was assumed that theone-way network delay is the same on both sides, i.e., the time takenfor a message from a master to a slave is the same as the time taken fortransit from a slave to the master. A small test for the effectivenessof the synchronization process was done. Because it was difficult tohave a centralized time for verification of synchronization process oftwo mobile phones, performance was measured in an alternative method asfollows.

First, the devices to be synchronized for the estimation were labeled asDevice 1 and Device 2. Device 1 was made to act as the master device andsynchronized making note of the offset. Device 2 was made to act as theslave device and synchronized making note of the offset. When the offsetvalue is 0, the clocks between the devices are perfectly synchronized.Steps 2 and 3 were repeated to verify zero offset value in bothdirections. The performance test was conducted on two pairs of devices:Nexus One vs Nexus One and Nexus One vs HTC Hero. Hero and Nexus One donot have a common hardware configuration. Hence, there was a tradeoff inthe network delay. Table 12 below shows the effectiveness of the clockafter synchronizing the two Nexus One mobile phones. Based on thisexperiment, it was found that the lower the offset time between devices,the greater the effectiveness. The offset varied between ±1 ms, whichshowed an effective synchronization of the clock times in both devices.The results for Nexus one vs Hero showed a similar behavior, but theround trip time (RI 1) between the devices was 79 ms and the offsetvaried between ±6 ms.

TABLE 12 Device RTT(ms) Offset(ms) 1 30 1 2 29 −1 1 30 −1 2 29 0 1 29 02 29 0 1 29 0 2 29 −1 1 29 1 2 29 0

A Bluetooth device can transmit data at a rate of 780 kbps according tothe IEEE 802.11 standards. The effective transmission and reduction ofthe RTT relies on the size of messages being sent and the processorcapability to handle multiple requests, interference and attenuation ofsignal. In the present investigation, the devices were placed as closeas possible to eliminate the delay introduced due to signal issues. Alsothe message size being transmitted was very small for a bandwidth of 780kbps. Therefore data loss due to retransmissions or transmission delaysdue to buffer overflows did not occur in the system. FIG. 20 gives thegeneralized packet structure of a Bluetooth data packet. During thesynchronization process, a timestamp value of long data type was beingsent back and forth during a time instant. Considering the payload sizeas 128 bits, approximate size of an individual data packet was about 32bytes. The mechanism works on simplex method sending data in onedirection at a time. Hence, the message size will not create issues intransmission.

The detection of heart sounds relies on the microphone's ability to pickup low-frequency, low-amplitude audio signals. Every mobile phone in themarket has different hardware specifications. Two brands of phones maynot have the same kind of hardware. However, a device with the bestmicrophone sensitivity could help in recording the heart beatsaccurately. Thus, a comparison of microphones on three devices was alsomade.

First, a test was performed to determine the frequency response of themicrophone on three devices, namely, Google's Nexus One, HTC Hero, andHTC Legend. The test occurred in a closed environment with ambient noiseand few audio activities. The three phones recorded the same audiosimultaneously. The recorded audio was processed in MATLAB to derive afrequency response. FIG. 21 shows the microphone frequency response forthree different mobile phones. FIG. 21 shows a flatter response curvefor Hero than Legend and Nexus One. Legend has the highest power in thelow frequency spectrum followed by Hero and Nexus One. Also, theoccurrence of peaks in the audio data due to external disturbance wasseen only in Hero and Legend, where as it was eliminated in Nexus One bythe hardware noise cancellation available on the phone. Thus, heartbeats were more accurately detected using Legend or Hero than Nexus One.

REFERENCES

The following documents and publications are hereby incorporated byreference.

Other Publications

-   Song, W., et al., “Next Generation 9-1-1 Proof-of-Concept System,”    SIGCOMM 2008.-   United Kingdom Department of Health—AMPDS Call Categorization    Version 11; Her Majesty's Stationery Office, April 2005.-   SAMU Online (http://www.samu-de-france.fr/en) NENA NG911 Project    Online (http://www.nena.org/n911-project)-   NENA i3 Technical Requirements Document,    (http://www.nena.org/standards/technical/voip/i3-requirements)-   Nada Hashmi, Dan Myung, Mark Gaynor, Steve Moulton, “A Sensor-based    web service-enabled emergency medical response system”, EESR'05,    June 2005.-   Fredrik Bergstrand, Jona Landgren, “Sharing Using Live Video in    Emergency Response Work”, Proceeding of the 6^(th) International    ISCRAM Conference—Gothenburg, Sweden, May 2009.-   Roman Belda, Ismael de Fez, Francisco Fraile, Vicot Murcia, Pau    Arce, Juan Carlos Guerri, “Multimedia System for emergency Services    over TETRA-DVBT Networks”, 34^(th) Euromicro Conference Software    Engineering and Advanced Applications, 2008.-   Reinvuo T., Hannula M., Sorvoja H., Alasaarela E., Myllyla R.,    “Measurement of Respiratory Rate with High-Resolution Accelerometer    and EMFit Pressure Sensor”, IEEE Sensor Application Symposium,    February 2006.-   Rendon D. B., Rojas Ojeda J. L., Crespo Foix L. F., Morillo D. S.,    Fernandez M. A., “Mapping the Human Body for Vibrations using an    Accelerometer”, Proceeding of the 29^(th) Annual Conference of the    IEEE EMBS. August 2007.-   Su S. W., Cellar B. G., Savkin A. V. Nguyen H. T., Cheng T. M., Guo    Y., Wang L., “Transient and Steady State Estimation of Human Oxygen    uptake based on Noninvasive Portable Sensor Measurements”, Medical    Biology Engineering Computation. March 2009.-   Phan D. H., Bonnet S., Guillemaud R., Castelli E., Pham Thi N.Y.,    “Estimation of Respiratory Waveform and Heart Rate using    Accelerometer”, 30^(th) Annual International IEEE EMBS Conference,    August 2008-   Richard Chipman, Roger Wuerfel, “Network Based Information sharing    Between Emergency Operations Center”, IEEE Conference on    Technologies for Homeland Security 2008.-   Green M. W., Sparks R., Pritchard D. A., “Real-time Video    Surveillance for First Responders in an Emergency Situation”, IEEE    International Conference on Security Technology 2008.-   Jeong O., Lee I. Shin-Gak K., “Consideration of Supporting the    Multimedia Emergency Services in VOIP”, International Conference on    Advanced Communication Technology 2009.-   Banitsas K., Pelegris P., Orbach T., Cavouras D., Sidiropoulos K.,    Kostopoulos S., “A Simple algorithm to Monitor HR for Real Time    Treatment Applications”, Proceedings of the 9^(th) 2008.-   J. Foo. C. Lim. and P. Wang; “Evaluation of Blood Pressure Changes    Using Vascular Transit Time”; Physiological Measurement, Vol 27, No,    8, 2006.-   K. Banitsas, P. Pelegris, T. Orbach, D. Cavouras, K. Sidiropoulos,    and S. Kostopoulos, “A simple algorithm to monitor hr for real time    treatment applications,” November 2009, pp. 1-5.-   J. Y. A. Foo, C. S. Lim, and P. Wang, “Evaluation of blood pressure    changes using vascular transit time,” Physiological Measurement,    vol. 27, no. 8, p. 685, 2006. [Online]. Available:    http://stacks.iop.org/0967-3334/27/i=8/a=003-   J. N. Cohn, S. Finkelstein, G. McVeigh, D. Morgan. L. LeMay, J.    Robinson, and J. Mock, “Noninvasive Pulse Wave Analysis for the    Early Detection of Vascular Disease.” Hypertension, vol. 26, no. 3,    pp. 503-508, 1995. [Online]. Available:    http://hyper.ahajournals.org/cgi/content/abstract/26/3/503-   J. Alfie, G. D. Waisman, C. R. Galarza, and M. I. Camera,    “Contribution of Stroke Volume to the Change in Pulse Pressure    Pattern With Age,” Hypertension, vol. 34, no. 4, pp. 808-812, 1999.    [Online]. Available:    http://hyper.ahajournals.org/cgi/content/abstract/34/4/808

1-24. (canceled)
 25. A method for capturing and transmitting chestsounds of a subject using a first mobile phone and a second mobilephone, wherein the first mobile phone comprises a microphone and iscapable of capturing and transmitting audio data, wherein the secondmobile phone comprises an embedded video camera capable of capturing andrecording video data with a lens and a flash and an application forestimating heart rate information using the camera and flash, andwherein the first mobile phone and the second mobile phone furthercomprise a WiFi receiver, a WiFi transmitter, and Voice over IP (“VOIP”)software, wherein the VOIP software comprises modified signalingprotocols which facilitate transmission of information in real-time,comprising: synchronizing time between the first mobile phone and thesecond mobile phone; placing the first mobile phone on the subject at anappropriate location close to the subject's chest; collecting audio datausing the first mobile phone that corresponds to chest sounds of thesubject; generating chest sounds information for the subject using theaudio data; activating the application for estimating heart rateinformation; and capturing the chest sounds information and the heartrate information; and using the modified signaling protocols of thefirst mobile phone and the second mobile phone to transmit one or moreof the chest sounds information and the heart rate information of thesubject to a recipient via an Internet-based network in real-time. 26.The method of claim 25, wherein the synchronizing time step utilizes aBluetooth based synchronization protocol.
 27. The method of claim 25,wherein the appropriate location close to the subject's chest is alocation close to the pulmonary valve of the subject's heart. 28-32.(canceled)
 33. The method of claim 25, wherein the first mobile phonecomprises an audio recording application.
 34. The method of claim 25,wherein the second mobile phone comprises a video recording application.35. The method of claim 25, wherein the placing the first mobile phoneon the subject step comprises pressing the microphone of the firstmobile phone against the chest of the subject.
 36. The method of claim25, wherein the appropriate location close to the subject's chest is onbare skin of the subject.
 37. The method of claim 25, wherein thesubject is in a seated position.
 38. The method of claim 25, wherein atleast one of the first mobile phone and the second mobile phone is aportable personal device carried by a user on a wrist or arm.